duckdb: Reset validity instead of filling#7365

Merged

myrrc merged 1 commit intodevelopfrom

myrrc/duckdb-validity

Apr 10, 2026

Contributor

myrrc commented Apr 9, 2026 •

edited

Loading

Reset validity if it's "all true" instead of filling manually.
Set validity to true if parent array is constant (slicing won't affect that).
Reset validity if buffer after copying is all ones or all zeroes.
Copy validity mask and count ones in one pass

myrrc requested a review from robert3005

April 9, 2026 15:29

myrrc added the changelog/performance label

myrrc force-pushed the myrrc/duckdb-validity branch from 1d39f2d to e5db410 Compare

April 9, 2026 15:30

myrrc enabled auto-merge (squash)

April 9, 2026 15:30

joseph-isaacs reviewed

View reviewed changes

vortex-duckdb/cpp/vector.cpp

myrrc force-pushed the myrrc/duckdb-validity branch from e5db410 to 30483d5 Compare

April 9, 2026 15:32

joseph-isaacs reviewed

View reviewed changes

vortex-duckdb/src/duckdb/vector.rs Outdated

joseph-isaacs reviewed

View reviewed changes

vortex-duckdb/src/exporter/mod.rs Outdated

joseph-isaacs reviewed

View reviewed changes

vortex-duckdb/src/exporter/vector.rs

joseph-isaacs reviewed

View reviewed changes

vortex-duckdb/src/exporter/vector.rs Outdated

myrrc force-pushed the myrrc/duckdb-validity branch from 30483d5 to f7f88fe Compare

April 9, 2026 17:31

myrrc requested a review from joseph-isaacs

April 9, 2026 17:33

joseph-isaacs reviewed

View reviewed changes

vortex-duckdb/src/exporter/vector.rs Outdated

codspeed-hq Bot commented Apr 9, 2026 •

edited

Loading

Merging this PR will not alter performance

✅ 1122 untouched benchmarks
⏩ 1455 skipped benchmarks¹

_{Comparing myrrc/duckdb-validity (7658463) with develop (ac87fe0)}

1455 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

joseph-isaacs reviewed

View reviewed changes

vortex-buffer/src/bit/buf.rs Outdated

joseph-isaacs reviewed

View reviewed changes

vortex-buffer/src/bit/buf.rs

joseph-isaacs requested changes

View reviewed changes

myrrc force-pushed the myrrc/duckdb-validity branch from f7f88fe to 73ac072 Compare

April 10, 2026 10:06

myrrc requested a review from joseph-isaacs

April 10, 2026 10:07

myrrc force-pushed the myrrc/duckdb-validity branch from 73ac072 to fc276d9 Compare

April 10, 2026 11:17

joseph-isaacs reviewed

View reviewed changes

vortex-duckdb/src/exporter/mod.rs Outdated

joseph-isaacs reviewed

View reviewed changes

vortex-duckdb/src/exporter/mod.rs Outdated


          reset validity instead of filling

Signed-off-by: Mikhail Kot <to@myrrc.dev>

myrrc force-pushed the myrrc/duckdb-validity branch from fc276d9 to 7658463 Compare

April 10, 2026 12:50

myrrc requested a review from joseph-isaacs

April 10, 2026 12:50

joseph-isaacs approved these changes

View reviewed changes

Contributor

joseph-isaacs left a comment

NICEE

myrrc merged commit a78897f into develop

63 of 64 checks passed

myrrc deleted the myrrc/duckdb-validity branch

April 10, 2026 13:23

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance